Papers with subtractive pruning-based probe
Low-Complexity Probing via Finding Subnetworks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to probing neural networks for linguistic properties are to train a shallow multi-layer perceptron (MLP) on top of the model's internal representations. |
| Approach: | They propose a subtractive pruning-based probe where they find an existing subnetwork that performs the linguistic task of interest. |
| Outcome: | The proposed probe achieves higher accuracy on pre-trained models and lower accuracy on random models, and better learning on its own. |